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Executive Summary 


As districts across the country search for ways to 
turn around their lowest-performing schools, a grow- 
ing number are partnering with external providers— 
often charter operators—to implement “restarts.” 
These providers receive operational autonomy and 
bring their own approach to schooling, positioning 
them to implement deep and meaningful changes. 
The school continues to serve the same community, 
and the provider contractually commits to raising 
student performance outcomes. 

Restarts are appealing, but do they deliver on 
their promise? This report draws on data from four 
restarts in Indianapolis Public Schools (IPS) to iden- 
tify successes and lessons learned for the field. 


About the Study 


This report analyzes how enrollment, demographic, 
and student performance data changed at the re- 
starts over time, and how those changes compared 
to other low-performing IPS schools in which the 
district was making significant improvement efforts. 
The research team also interviewed representatives 
from the district, restart operators, and The Mind 
Trust, a local education organization that played a 
key role in developing the Innovation Schools frame- 
work under which restarts operate. These interviews 
furthered the research team's understanding of the 
processes, challenges, and opportunities that re- 
starts offer. 


Findings 
The analyses and interviews described above led to 
five findings: 


1. Enrollment trends offer positive signs. Increases 
in both reenrollment rates and overall enrollment 
at the restarted schools suggest strong parent 
satisfaction. Moreover, the data show that the 
schools are mostly enrolling the same mix of stu- 
dents as before the restart. 


2. Performance data from the longest-running 
restart suggest the possibility of large gains 
over time. The impact of IPS’s restarts on student 
performance outcomes has been mixed overall. 
However, at the first restart, begun in 2015-16, 
the average schoolwide growth score has in- 
creased each year in both ELA and math. Over 
three years, the score has increased by a total 
of more than 15 percentiles in both subjects. 


3. Anecdotal evidence highlights the difficulty 
of restarting struggling schools. The first three 
restarts from this study encountered substantial 
community concern, causing leaders to spend 
more time and energy than they anticipated com- 
municating with families and community partners 
and hiring new staff. IPS has since hired a family 
and community engagement manager to help ele- 
vate community voice in the restart process. Even 
if they are serving students with a similar level of 
need, many charter operators acknowledge that 
restarting a chronically low-performing school 
entails challenges that starting a new school from 
the ground up does not. 


4. Access to facilities provides districts a power- 
ful enticement. School operators noted that the 


restart model offered them access to a low-cost 
facility. Without that, they may have not been 
able to open a school at all; with it, they were will- 
ing to pursue the difficult option of restart. 


5. Disconnected data systems hamper evaluation. 
District and charter operators generally “own” 
their data and seldom use the same data systems, 
making it difficult for operators—both district and 
charter—or researchers to evaluate initiatives. 
There are clear gaps in data systems as districts 
cease to be the sole providers of public education 
ina city. 


Recommendations 


The findings support three recommendations for 


districts, states, and policymakers: 


1. Engage the school community early and 
honestly. So long as districts continue to pursue 
restarts, they should put as much thought and 
energy as possible into real and authentic com- 
munity engagement from the very beginning. 


2. Use your assets. Access to district resources, 
such as facilities and local tax dollars, may push 
high-potential operators to work with districts 
in new and impactful ways, including taking re- 
sponsibility for the difficult task of transforming 
chronically low-performing schools. 


3. Build infrastructures that can support systems 
of schools across a city. As more operators share 
responsibility for enrolling students in the same 
geography, city leaders must update their data in- 
frastructures to serve students and families more 


seamlessly. 


As districts struggle to turn around chronically low- 
performing schools, they frequently fail—in part 
because they fail to fundamentally change how the 
school operates. But districts also want to avoid 
closing struggling schools; neighborhood schools 
often sit at the heart of a community, and nearby 
excellent schools rarely, if ever, have enough empty 
seats to accommodate displaced students. 

As districts across the country search for ways to 
turn around their lowest-performing schools, a grow- 
ing number are partnering with external providers— 
often charter operators—to implement “restarts.” 
These providers receive operational autonomy and 
bring their own approach to schooling, positioning 
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them to implement deep and meaningful changes. 
The school continues to serve the same community, 
and the provider contractually commits to raising 
student performance outcomes. 

Restarts are appealing, but do they deliver on 
their promise? This report draws on data from four 
restarts in Indianapolis Public Schools (IPS) to iden- 
tify successes and lessons learned for the field. The 
research base is thin, and the results mixed (see 
Appendix A—Literature Review, page 24). This 
report aims to add to the field’s understanding by 
drawing on data from four Indianapolis restarts to 
evaluate their impact on student enrollment, mobil- 
ity, demographics, and performance. 


Sparking Innovation in 
Indianapolis Public Schools 


Indianapolis Public Schools (IPS) struggled to sup- 
port high student achievement for many years. 
In 2013-14, of the 75 IPS schools that the state 
rated, 54 percent earned a D or worse.! Of those, 11 
schools had received an F rating for two consecutive 
years, and their student test scores demonstrated 
low, no, or negative growth.? Meanwhile, the threat 
of state takeover loomed. The previous year, the 
Indiana Department of Education handed responsi- 
bility for four IPS schools over to private operators 
after the district failed to turn them around.° 

It was against this backdrop that IPS’s leadership 
began to change. IPS school board elections in 2012 
and 2014 ushered in a number of new members. In 
2013, Dr. Lewis Ferebee replaced the superintendent 
of eight years.* In March 2014, state legislators 
passed House Bill 1321, which allowed IPS to create 
Innovation Network Schools with greater flexibility 
to operate differently. With the support of the school 
board and other partners, Ferebee put the law into 


action.® 


Innovation Network Schools Offer 
Freedom and Support 


Innovation Network Schools include district and 
charter schools, and both benefit from participat- 
ing.® District schools receive greater autonomy, 
including autonomy over staff compensation, the 
school calendar, staffing positions, hiring criteria, and 
instructional practices. Meanwhile, Innovation status 
gives charter schools the ability to establish a more 
formal relationship with the district and gain access 
to several key district resources, including transpor- 
tation and food services, and sometimes facilities 
as well. All Innovation schools, including district and 
charter, must organize a nonprofit board to oversee 
school operations and abide by all local, state, and 
federal laws, including those regarding special edu- 
cation and non-discrimination. 

Innovation schools “combine the freedoms and 
flexibilities of successful autonomous schools with 


the financial support and services of a district 


school,” said Brandon Brown, CEO of The Mind 
Trust, an Indianapolis education organization that 
played a key role in developing the Innovation 
schools idea.’ At the same time, student perfor- 
mance at Innovation schools falls under the district 
for state accountability purposes, encouraging IPS 
to collaborate with strong operators. 

There are four kinds of Innovation schools, which 
vary in terms of management and their pathway to 
Innovation school status (see Figure 1, page 5): 


1. Existing or new charter schools can apply to be- 


come Innovation schools; 


2. New IPS district schools can apply to start as In- 
novation schools; 


3. Existing IPS district schools can convert to Inno- 
vation schools if they demonstrate strong leader- 
ship capacity and the current staff members want 


to convert; and 


4. The district can partner with an external opera- 
tor to restart a chronically low-performing district 


school, which then becomes an Innovation school. 


The number of Innovation schools in IPS grew from 
five schools in 2015-16 to 20 schools in 2018-19 
(see Figure 2, page 5). 


Restart Schools 


IPS selected some of the district’s lowest-perform- 
ing schools for restart. By contracting with high- 
potential operators, IPS aimed to turn the schools 
around and give their students better options, while 
also allowing the district to maintain ultimate over- 
sight of the schools. 

Restarts differ most notably from other Innova- 
tion schools in three areas: 


* Facilities. Restart charter operators have access 
to district facilities at the same cost as district- 
operated schools; other charters with Innovation 
status do not necessarily get that. 

%* Zoned charters. Unlike other charter schools in 
Indiana, restarts must give preference to students 
within residential boundaries. Hence, rather than 
opting into a restarted school, students zoned for 


FIGURE 1. OVERVIEW OF INNOVATION SCHOOLS 


Key School 


Characteristics 


Benefits to 
operator 


Charters 


% New or existing 
charter schools 


Type of Innovation School 


New District 


* New district 
schools 


District 
Conversions 


%* Existing district 
schools, converted 
of own choosing 


Restarts 


%* Chronically low-performing 
district schools 
%* External operator runs school 


%* Partnership with 
district, including 


* Greater 
autonomy 


* Greater 
autonomy 


* Access to district facilities 
without lease or building costs 


access to some 
district services, like 
low-cost facilities 
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* Access to IPS enrollment/ 
students 
%* Accountability clock reset 


FIGURE 2. GROWTH OF INNOVATION SCHOOLS 
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the school are automatically enrolled unless they 
opt out. If space remains after accounting for 
zoned students, students zoned for other schools 
may also enroll. 

* Accountability reset. Since restarts are ulti- 
mately new schools, the accountability clock 


resets—meaning state ratings before the restart 
do not apply. Moreover, beginning in 2019-20, 
restarts will not receive a rating for the first three 
years unless they request otherwise.® However, the 
restarted school must provide a timeline that out- 
lines annual targets for academic improvements. 


serve many of the same students. 


Analysis: Making Sense of the Data 


Early research on Innovation schools shows promis- 
ing results. Stanford’s Center for Research on Edu- 
cation Outcomes (CREDO) compared the learning 
growth that Indianapolis students attending differ- 
ent kinds of schools—including Innovation schools 
and traditional public schools—made relative to 
state averages.® The results show that in the most 
recent school year included in the study (2016-17), 
students attending traditional public schools made 
significantly less growth than the state average 
learning gains. In contrast, students attending In- 
novation schools made learning gains similar to the 
state average.!° The published research, however, 
lumps all Innovation schools together in one cat- 
egory, so it’s not possible to isolate the impact of 
restarts separately. 

The rest of this report does just that, focusing 
on the first four Innovation restart schools—Phalen 
Leadership Academy @ 103 (PLA @ 103), Global 
Preparatory Academy (Global Prep), Kindezi Acad- 
emy (Kindezi), and Ignite Achievement Academy 
(Ignite). It aims to identify successes and lessons 
useful for future restarts in IPS and elsewhere (for 
more on these four schools, see Figure 3, page 7 


WHAT?’S DIFFERENT INSIDE A RESTARTED SCHOOL? 


According to the operators featured in this report, the answer is—everything. The restart operators 
not only brought in new leadership teams and governance structures, but also renovated and re- 
branded buildings, introduced new curricula (including a bilingual program at one), adjusted the school 
day and calendar, and implemented new discipline policies and practices. Though existing teachers 
had an opportunity to re-apply for their positions, few did, and the restart operators hired a mostly 
new—if not entirely new—teaching staff. In addition, the operators invited parents into their buildings 
to see how they had changed in an effort to reset their relationship with families. Ultimately, the op- 
erators all aimed for the schools to both look and feel completely different, even as they continued to 


and “The Same, but Different,” page 7)."' The study 


focuses on three research questions: 


1. How have restarts affected student 
enrollment and mobility? 


2. How have restarts affected student 
demographics? 


3. How have restarts affected student 
performance outcomes? 


To answer these questions, the research team re- 
ceived various student-level data from the Indiana 
Department of Education, including enrollment, 
demographic, and student performance data. The 
team then analyzed the data to observe how the 
metrics changed at each school since the restart 
began. In addition, the team compiled data across 
restart schools and compared them to other low- 
performing district schools where IPS was making 
significant improvement efforts, including four “pri- 
ority schools” and four schools belonging to one of 
the district’s transformation zones (see “About the 
Comparison Schools,” page 8). The rest of this sec- 


tion presents the findings from those analyses. 


FIGURE 3. INDIANAPOLIS RESTARTS AS OF 2017-18 SCHOOL YEAR 


School Name Previous School Grades Served First Year of Restart 
PLA @ 103 Francis Scott Key 103 Pre K-6* 2015-16 
Global Prep Riverside 44 Pre K-6 2016-17 
Kindezi Joyce Kilmer 69 K-6 2016-17 
Ignite Elder Diggs 42 K-6 2017-18 


*In 2016-17, PLA @ 103 also served grade 7 (but not in subsequent years). 


THE SAME, BUT DIFFERENT 


Though all are restarts, the governance structure and transition process differed for the schools fea- 
tured in this report (see table below). 

The first restart, PLA @ 103, began in 2015-16 and is technically still a district school using the 
same school ID as the school it replaced, Francis Scott Key 103. In contrast, the other restarts are all 
charter schools that the mayor's Office of Education Innovation authorizes. As part of PLA’s agree- 
ment with IPS, it receives a management fee to operate the school in addition to its regular per-pupil 
funding, something no other restart receives (or will receive). 

Global Prep and Kindezi are phasing in as charter operators. Although they operate all grades 
within their respective buildings, only grades K-2 restarted as charter schools with a new school ID, 
because of restrictions in the charter contracts they received from their authorizer before IPS se- 
lected them as restart operators. IPS now has a contract with them to run the remaining grades as 
district schools under the original school IDs. Each year the charter side of the schools expands by 
one grade, while the district side of the schools shrinks by one grade so that all grades will eventually 
fall under the charter operator. But both IPS and the operators say the division between the district 
and charter sides of these schools is largely on paper, with little difference in practice between the 
two school sides. Both PLA @ 103 and Ignite restarted the whole school at once. 


School Name Governance Transition Process 
PLA @ 103 Contract Whole school 

Global Prep Charter Phase-in 

Kindezi Charter Phase-in 

Ignite Charter Whole school 


ABOUT THE COMPARISON SCHOOLS 


For each of the analyses in this report, we compared the results of the restarted schools to Priority 
schools and schools belonging to the district’s Transformation Zones.% 


* “Priority school” is a federal designation meant to include the lowest 5 percent of Title | schools in 


each state, as defined by the state. All the Priority schools analyzed in this study had received an 

F rating from the state for two consecutive years and demonstrated low, no, or negative growth. 
These schools receive greater attention from the district, and often additional operating flexibility 
and resources to help them improve. Our comparison group includes the four schools designated 
as Priority schools beginning in the 2015-16 school year serving grades K-8."4 All four of the re- 
started schools in this study were also designated as Priority schools (though they are not included 
in the Priority schools analysis). 


* IPS’s Transformation Zones include a struggling high school and a small set of feeder schools 


for which the district develops and implements a transformation strategy focused on district-led 
turnaround supports (rather than contracting with third parties to run the school). It launched 

in 2015-16 and represents the other major turnaround intervention IPS implemented aside from 
the Innovation Schools Network. Our analysis includes four Transformation Zone schools serving 


grades K-8 during the study period.® 


RESEARCH QUESTION 1. 
How have restarts affected student 
enrollment and mobility? 


Restarts aim to give students who are attending 
or zoned for a failing school a better education. 
To meet that goal, however, those same students 
must actually enroll in the restarted school. We 
therefore examined the extent to which students 
eligible to reenroll at a restarted school did so, 
and how reenrollment rates have changed at the 
restarted schools over time.'® 


RESTARTS REACHED JUST OVER 
HALF OF ELIGIBLE STUDENTS 


For this measure, eligible students are defined as 
students who were enrolled at the school right be- 
fore the restart began and were not in a terminal 
grade (the last grade the school serves). Restart 
success can be measured in part by the percentage 
of eligible students who reenrolled at the restarted 
schools and could therefore benefit from the in- 


tervention. Across the four restarts, 55 percent of 


eligible students returned to the restarted school in 
year 1. As Figure 4 (page 9) shows, however, reen- 
rollment rates were much lower at PLA @ 103 (35.8 
percent) compared with the other three restarts 
(57.1 percent to 64.1 percent). 

While a 55 percent reenrollment rate may seem 
low, looking back at the three years leading up to 
each restart shows that it was about average, and 
that reenrollment rates actually increased at Global 
Prep and Kindezi in year 1 (see Figure 5, page 10). 
Meanwhile at PLA @ 103, the reenrollment rate 
decreased by less in year 1 than it had the previous 
year. 


MORE RECENT ENROLLMENT TRENDS POSITIVE 


Looking beyond year 1, reenrollment rates have in- 
creased at all three of the restarts that have at least 
two years of data; at two of the restarts, the rates 
increased by double digits. Reenrollment averaged 
47 percent in the two years before Phalen @ 103 
restarted. It surpassed 70 percent for each of the 
two most recent years (2016-17 and 2017-18) — 

an increase of more than 20 points (and double its 


FIGURE 4. REENROLLMENT OF ELIGIBLE” STUDENTS IN RESTARTED SCHOOLS, YEAR 1 


Eligible Students Percentage of 
Students Eligible WhoDidReenroll Eligible Students 
Baseline to Reenroll at End in Year 1 of Who Did Reenroll in 

Year of Baseline Year Restart Year 1 of Restart 
Francis Scott Key / PLA @ 103 2014-15 299 107 35.8% 
Riverside 44 / Global Prep 2015-16 401 257 64.1% 
Joyce Kilmer 69 / Kindezi 2015-16 296 177 59.8% 
Elder Diggs 42 / Ignite 2016-17 354 202 57.1% 
All Restarts 1,350 743 55.0% 


“The percentage of students who were enrolled at the school in the baseline year and were not in a terminal grade. 


year 1 reenrollment rate). At Global Prep, reenroll- a negative trend in the years leading up to the re- 

ment rates rose from about 50 percent to more than start (see Figure 7, page 11). In contrast, enrollment 

65 percent. These increases were larger than both in Transformation Zone and Priority schools has 

comparable schools and district-wide trends over the largely remained steady for schools that remained 

study period (see Figure 6, page 11). open. (One school in the Transformation Zone re- 
At the same time, overall enrollment has in- started in 2018-19, and another closed.) 


creased at all four of the restarted schools, reversing 


SUMMARY: IMPACT ON STUDENT ENROLLMENT AND MOBILITY 


%* Across the restarts, 55 percent of eligible students reenrolled in year 1, about equal to schools’ 
average reenrollment rate in the three years preceding restart. 

% Reenrollment rates at all three restarts with two or more years of data have increased compared 
with the three years leading up to the restart, two of them by double digits. 

% Overall enrollment at each of the restarted schools has increased, reversing a decline in enrollment 
before the restart. 


FIGURE 5. PERCENTAGE OF ELIGIBLE STUDENTS REENROLLING YEAR-OVER-YEAR IN RESTARTS VS. COMPARISON SCHOOLS, 
THREE YEARS PRE-RESTART THROUGH 2017-18 


Pre-Restart Post-Restart Change 


3-Yr Pre- Most 
Restart Recent 


-3 Years -2 Years -1 Years ; Year 
N/A* 


52.6% 41.0% 


Francis Scott Key / PLA @ 103 70.3% 


N=344 N=410 
49.3% 49.8% 51.2% 
Riverside 44 / Global Prep 65.9% 
N=430 N=404 N=373 
55.0% 52.8% 54.1% 
Joyce Kilmer 69 / Kindezi 57.6% 
N=422 N=390 N=338 
63.0% 56.7% 60.0% 
Elder Diggs 42 / Ignite 59.9% 57.1% -2.8% 
N=511 N=492 N=492 
56.2% 53.2% 51.9% 
All Restarts 53.8% 55.0%** 1.3% 
N=1363 N=1630 N=1613 
N/A* 66.3% 63.7% : 67.9% 71.5% 
Transformation Zone Schools 65.0% 71.5% 6.5% 
N=2295 N=2358 N=2045 N=1943 N=2203 
N/A* 65.4% 62.6% 63.8% 65.8% 67.5% 
Priority Schools 64.0% 67.5% 3.4% 
N=1808 N=1883 N=1800 N=1673 N=1653 
*We did not have access to the 2011-12 enrollment data needed to calculate these reenrollment rates. 
**Based on most recent year for which all schools have data (Y1), rather than an average of the rows above. 
Change > 0, <=5 pts. Change > 5, <=10 pts. Change > 10, <=15 pts. Change >15 pts. 
Ke 
i“ Change < 0, >=-5 pts. Change < -5, >=-10 pts. Change < -10, >=-15 pts. Change <-15 pts. 
FIGURE 6. PERCENTAGE OF ELIGIBLE STUDENTS REENROLLING AT THE SAME SCHOOL YEAR-OVER-YEAR, 
ALL PUBLIC SCHOOLS IN IPS, 2013-18 
yeh he} 2014 yA0h KR) 2016 2017 2018 
Not avail. 69.7% 70.9% 71.4% 73.2% 74.1% 


Increase compared to previous year 
Decrease compared to previous year 


FIGURE 7. CHANGE IN STUDENT ENROLLMENT, RESTARTS VS. COMPARISON SCHOOLS, THREE YEARS PRE-RESTART THROUGH 2018-19 


Pre-Restart Post-Restart 


-3 Years -2Years’ -1 Years Year 1 Year 2 Year 3 Year 4 


Francis Scott Key / PLA @ 103 
Riverside 44 / Global Prep 494 421 421 
Joyce Kilmer 69 / Kindezi 459 376 323 
Elder Diggs 42 / Ignite 


All Restarts 
1949** 


Transformation Zone Schools 


3-Yr Pre- 
Restart 
Avg. 


Change 


Most 


Recent 


Year 


Change 


445 


673 


386 


395 


1949** 


Priority Schools 1925 2112 1933 1716 


1716 


*Based on most recent year for which all schools have data (Y1), rather than an average of the rows above. 


**In 2018-19, one Transformation Zone school (Wendell Phillips) restarted as an Innovation school and another (George Washington Jr. High School) closed, accounting 


for much of the decline in enrollment across Transformation Zone schools that year. 


Increase in enrollment compared to previous year 
Decrease in enrollment compared to previous year 


RESEARCH QUESTION 2. 

How have restarts affected student 
demographics? 

If restarts continue to serve the same students and 
neighborhoods, they should enroll a similar mix of 
students as the previous school unless the neighbor- 
hood is also changing. Significant changes in sub- 
group representation would suggest that the restart 
may be attracting particular types of students, or 
that other groups may be discouraged from enroll- 
ing. Hence, we examined how the students who 
reenrolled at the restarted school in the first year of 
the restart resembled (or did not resemble) the mix 
of students enrolled the year before the restart. We 
also analyzed how student demographics at the re- 
started schools changed over time. 


STAYERS MOSTLY RESEMBLE STUDENT BODY 
BEFORE RESTART 


Eligible students who reenrolled in the restarted 
schools (“Stayers”) largely exhibited the same char- 
acteristics as all students eligible to reenroll at the 
school the year before the restart (see Figure 8, 
page 13), indicating that the restarts were not more 
likely to recruit or retain any particular kinds of 
students. The percentage of Stayers who identified 
as black or Hispanic, qualified for free or reduced- 
priced lunch, or had limited English proficiency were 
all within 1.6 percentage points of the broader stu- 
dent body in the baseline year. 


DISTRICT RETAINED SPECIAL EDUCATION 
PROGRAMS AT TWO RESTARTS 


The data contain, however, one notable exception: 
Across the restarted schools—largely due to two of 
the schools—just 13.2 percent of Stayers were in 
special education, compared with 17.8 percent of all 
students enrolled in the baseline year—a difference 
of 4.6 percentage points. That difference was more 
than twice that of any other subgroup (see Appen- 
dix B, page 25). At PLA @ 103, students in special 
education made up 22 percent of students eligible 


to reenroll the year before the school restarted, but 
they were just 11 percent of Stayers—a decrease of 
50 percent. Similarly, students in special education 
at Ignite represented 22 percent of students eligible 
to reenroll, but only 13 percent of Stayers—a de- 
crease of 9 percentage points. 

According to district staff, the shift mostly 
reflects the fact that IPS retained control of the 
special education programs at those schools— 
self-contained classrooms serving students with 
some of the greatest needs, including life skills 
classes for older students and developmental 
pre-K—rather than any decisions the new operators 
made. One such program moved from Phalen @ 103 
to another IPS school. Meanwhile, the two programs 
at Ignite continued in the building, but remained 
part of IPS’s local education agency, the legal entity 
responsible for the students and implementing their 
individualized education programs. 


CHANGES IN STUDENT DEMOGRAPHICS 
AT RESTARTS MOSTLY FOLLOW CITYWIDE 
TRENDS OVER TIME 


Figure 9 (page 13) considers how student demo- 
graphics have changed throughout the restarted 
schools over time. It shows where student enrollment 
at each restart and comparison group rose or fell by 
more than 10 percentage points beyond the change 
across all public schools located in the IPS bound- 
aries.” For example, at Kindezi, the percentage of 
black students enrolled in the three years prior to 
the restart through 2018-19 decreased by 7.3 per- 
centage points, while it decreased by 4 percentage 
points across all public schools in the IPS boundaries 
over the same period, for a net change of 3.3 per- 
centage points (see Appendix C1, page 26). Since 
3.3 is less than 10, an “o” appears in that cell. 

Changes in student demographics at restarted 
schools closely track other public schools in the area, 
as is evident in the many cells marked “o” in Figure 9 
(see also Appendix C, page 26).1® 


FIGURE 8. STUDENT CHARACTERISTICS ACROSS ALL RESTARTS, 
ALL STUDENTS IN THE BASELINE YEAR VS. REENROLLED STUDENTS (“STAYERS”) IN YEAR 1 


Percent of Percent of 
All Students Enrolled Reenrolled Students 


Baseline Year (“Stayers”), Year 1 Change 
Student Subgroup (N=1303) (N=743) (Percentage Points) 


Black 


Hispanic 
Free & Reduced-Price Lunch 84.5% 86.1% 1.6% 
Limited English Proficiency 6.2% 6.6% 0.4% 


Special Education 


Notes 

* If demographic data were not available for a particular student in the baseline year, we used the demographic data for the 
same student from year 1. 

* Number of students captured in Figure 8 does not match the number in Figure 4 because demographic data were not 
available for some students for whom we had enrollment data. 


FIGURE 9. CHANGE IN STUDENT DEMOGRAPHICS AT SCHOOL ABOVE CHANGE ACROSS ALL PUBLIC 
SCHOOLS IN THE IPS BOUNDARIES, THREE YEARS PRE-RESTART THROUGH 2018-19 


Francis Scott Key / PLA @ 103 ro) o ° ) 

Riverside 44 / Global Prep fo) 

Joyce Kilmer 69 / Kindezi ° ° ° ° ° 

Elder Diggs 42 / Ignite ) ro) fo) fc) fc) 

All Restarts fc) co) ° ° fc) 
Transformation Zone Schools* ro) fo) Co) lt eid 
Priority Schools ° ° ° ° ° 


*In 2018-19, one Transformation Zone school (Wendell Phillips) restarted as an Innovation school and another (George 
Washington Jr. High School) closed, so calculations for Transformation Zone schools in 2018-19 do not include them. 


Change in student representation increased >=10 pts. above 
g p p 
change across all public schools in the IPS boundaries 


Change in student representation decreased >=10 pts. above 
change across all public schools in the IPS boundaries 


But there are two notable exceptions. First, the 
percentage of students in special education has 
decreased by a larger amount at PLA @ 103 (see 
Figure 9, page 13, and Appendix C, Figure C9, page 
30). As noted earlier, however, that decrease likely 
reflects the fact that the school previously housed 
a center for students with special needs. 

Second, at Global Prep, school leaders have in- 
tentionally recruited Hispanic students to attend its 
dual-language program. As a result, the percent- 
age of Hispanic and ELL students has increased 
at Global Prep by larger amounts than at other 
schools, while the percentage of black students has 
decreased by a larger amount (see Figure 9, page 
13 and Appendix C, page 26). Since overall student 
enrollment also increased at Global Prep, however, 
the number of black students did not change as 
dramatically. In fact, more black students enrolled in 
the first two years following the restart than in the 
baseline year. The percentage of students qualify- 
ing for free and reduced-priced lunch also increased 
substantially at Global Prep as student enrollment 
grew and student demographics have shifted more 
broadly. 


for the 2019-20 school year.!® 


A LINGERING QUESTION: ARE NEIGHBORHOOD STUDENTS 
ENROLLING IN THE RESTARTS AT THE SAME RATES? 


The research team had also hoped to answer another question related to enrollment and mobility: 
Are the restarted schools continuing to serve students from the same neighborhoods? School choice 
options for students residing in IPS have been increasing over the past decade, however. In 2011-12, 
three years before the first restart began, there were just 16 charter schools in Indianapolis. IPS 

also had about a dozen magnet schools. By 2019-20, families could choose from among 38 charter 
schools, 18 Innovation schools, and 32 district “choice” programs open to students across the district 


With the increased opportunities for students to go somewhere other than their zoned school, dif- 
ferences in the percentage of students from a particular catchment area enrolled in a neighborhood 
school may not reflect changes in parents’ attitudes about the school, but rather changes in the op- 
tions available to them. Our analysis of Research Question 2 shows, however, that with few excep- 
tions, the demographic make-up of the students enrolled at the restarted schools is similar to that 
before the restart, suggesting that the schools largely pull students from the same neighborhoods. 


SUMMARY: RESTARTS AND 
STUDENT DEMOGRAPHICS 


%* Stayers largely resembled the demo- 
graphic make-up of all students eligible 
to reenroll at the restarted schools in the 
baseline year. 

%* Changes in student demographics at 
restarted schools have largely followed 
citywide trends, except when the dis- 
trict moved or retained responsibility for 
centers for students with special needs 
that had been located within a restarted 
school, or when the school intention- 
ally recruited students of a different 
demographic. 


RESEARCH QUESTION 3. 
How have restarts affected student 
performance outcomes? 


The ultimate goal of any restart is to improve the 
performance outcomes of the students who attend 
the school above what they would have achieved 
academically at the original school. To that end, the 
research team analyzed student growth percentiles 
(SGP). SGPs assign students a score from 1 to 99, 
indicating how much growth the student made rela- 
tive to students performing similarly at the end of 
the previous school year (per the state exam). For 
example, a student who receives an SGP of 50 made 
more growth than 50 percent of students perform- 
ing similarly at the end of the previous school year. 


Indiana students receive an SGP in grades 4-8 so 
long as they take the state exam each year. 

For this study, we analyzed how the mean SGP?° 
for students who reenrolled at the restarted schools 
(“Stayers”) compared to those who were eligible to 
stay, but chose to leave (“Leavers”), as well as how 
the mean SGP changed for all students attending 
the schools year-over-year.” 


STAYERS AND LEAVERS BOTH 
DEMONSTRATED GROWTH 


Figure 10 shows how mean SGPs changed among 
Stayers and Leavers with SGP data in the base- 
line year and year 1. Stayers’ mean SGP generally 
increased in year 1, improving an average of 6.7 


FIGURE 10. CHANGE IN MEAN SGP, STAYERS V. LEAVERS, BASELINE TO YEAR 1 


Change in Mean SGP from Baseline to Year 1 


ELA 


Stayers 


PLA @ 10 Leavers 


Stayers-Leavers 


Stayers 


Leavers 


Global Prep 


Stayers-Leavers 


Stayers 


Kindezi Leavers 


Stayers-Leavers 


Stayers 


Ignite Leavers 


Stayers-Leavers 


Stayers 


All Restarts Leavers 


Stayers-Leavers 


Math 


Note. “Stayers” defined as students eligible to reenroll at the end of the baseline year who did reenroll. To be included in 

the dataset, Stayers also had to have an SGP in the baseline year and in year 1. In addition, their SGP score in year 1 had 

to be linked to the restarted school. In contrast, “Leavers” were eligible to reenroll at the end of the baseline year, but did not 
reenroll. They also had to have SGP data for the baseline year and year 1, though their SGP in year 1 could not be linked to 


the restarted school. 


Change is significant at p<0.05 


Change within group is positive / Change of Stayers is larger than change of Leavers 


Change within group is negative / Change of Stayers is less than change of Leavers 


percentiles in ELA and 3.3 percentiles in math across 
the restarts, although only the gains in ELA were 
statistically significant. Leavers made even larger 
gains, however, improving by an average of 15.4 and 
16.6 percentiles in ELA and math, respectively; both 
were statistically significant. The difference between 
the gains Leavers made compared to Stayers (the 
difference-in-difference) was also statistically sig- 
nificant, though not the difference in ELA gains. 
(For additional detail, see Appendices D1 and Da, 
page 31.) 

Analyzing data for Stayers and Leavers with 
two years of post-restart growth data”? shows that 
both groups improved their mean SGPs in ELA and 
math, though the gains were larger (and significant) 


for Leavers in ELA and larger (and significant) for 
Stayers in math (see Figure 11, below). Overall, 

the impact of restarts on mean SGP of Stayers is 
unclear. Stayers demonstrated growth after the re- 
start, but so did Leavers, and the difference between 
their growth (difference-in-differences) was not sig- 
nificant after two years. (For additional detail, 

see Appendices D3 and D4, page 32). 


SCHOOLWIDE GROWTH ACROSS RESTARTS 
UNEVEN, WITH STRONGEST POSITIVE RESULTS 
AT PLA @ 103 


Year-over-year changes in mean SGP at the school 
level were also mixed (see Figures 12 and 13, pages 
17 and 18, respectively). The strongest positive 


FIGURE 11. CHANGE IN MEAN SGP, STAYERS VS. LEAVERS, BASELINE TO YEAR 2 


Change in Average SGP from Baseline to Year 2 
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All Restarts Leavers 
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Note. “Stayers” defined as students eligible to reenroll at the end of the baseline year who did reenroll. To be included in the 
dataset, Stayers also had to have an SGP in the baseline year, year 1, and year 2. In addition, their SGP scores in year 1 and 
year 2 had to be linked to the restarted school. In contrast, “Leavers” were eligible to reenroll at the end of the baseline year, 
but did not reenroll. They also had to have SGP data for the baseline year, year 1, and year 2, though their SGP scores in 
year 1 and year 2 could not be linked to the restarted school. 


Change is significant at p<0.05 


Change within group is positive / Change of Stayers is larger than change of Leavers 


Change within group is negative / Change of Stayers is less than change of Leavers 


FIGURE 12. CHANGE IN MEAN ELA SGP, RESTARTS VS. COMPARISON SCHOOLS, 
THREE YEARS PRE-RESTART THROUGH 2017-18 


Francis Scott Key / 
PLA 


Pre-Restart 


Post-Restart 


Change Over Time 


3-Yr 
Baseline 
Average 


Most Recent 


Restart Year Change 


N=109 N=85 N=69 
Riverside 44 / 45.2 36.7 43.6 
Global Prep N=120 N=110 N=144 
Joyce Kilmer 69 / 41.6 42.3 40.9 
Kindezi N=142 N=116 N=113 


Elder Diggs 42 / 
Ignite 


All Restarts 


Transformation Zone 


Schools N=1396 N=1311 N=1105 
41.4 49.0 49.9 
Priority Schools 
N=629 N=593 N=624 


*Based on most recent year for which all schools have data (Y1), rather than an average of the rows above. 


Change > 0, <=5 pts. 


Change > 5, <=10 pts. 


Change > 10, <=15 pts. 


Change >15 pts. 


Key 


Change < 0, >=-5 pts. 


A 


Change < -5, >=-10 pts. 


Change < -10, >=-15 pts. 


Change <-15 pts. 


FIGURE 13. CHANGE IN MEAN MATH SGP, RESTARTS VS. COMPARISON SCHOOLS, 
THREE YEARS PRE-RESTART THROUGH 2017-18 


Pre-Restart Post-Restart 


Elder Diggs 42 / 
Ignite 


All Restarts 


Francis Scott Key / 

PLA N=112 N=83 N=70 

Riverside 44 / 41.2 27.1 34.1 

Global Prep N=122 N=110 N=145 

Joyce Kilmer 69 / 42.3 35.1 29.7 

Kindezi N=146 N=124 N=117 
32.8 30.0 40.1 


Change Over Time 


3-Yr 


Baseline Most Recent 


Average Restart Year Change 


Transformation Zone 40.5 34.4 38.1 
Schools N=1414 N=1349 N=1164 
41.7 49.6 43.7 
Priority Schools 
N=620 N=613 N=635 


*Based on most recent year for which all schools have data (Y1), rather than an average of the rows above. 


Change > 0, <=5 pts. 


Change > 5, <=10 pts. 


Change > 10, <=15 pts. 


Change >15 pts. 


Key 


Change < 0, >=-5 pts. 


A 


Change < -5, >=-10 pts. 


Change < -10, >=-15 pts. 


Change <-15 pts. 


results come from PLA @ 103, where the restart has 
been in place longest. By 2017-18, the mean SGP 
there increased by more than 17 percentiles in both 
ELA and math compared with the three-year aver- 
age pre-restart.*? Global Prep also made large gains 
in math in 2017-18, increasing more than 17 percen- 
tiles. After strong ELA growth in year 1, however, 
Global Prep’s mean SGP dropped from 52.7 to 40.3. 
Ignite made modest gains in ELA, but experienced 
declines in math growth since restarting, and growth 
declined at Kindezi in both subjects.”4 Similarly, 
Transformation Zone and Priority schools have had 


mixed results. 


BALANCING IMPACT AND COST 


clear when. 


Impact on student outcomes is always the top priority when developing a school improvement initia- 
tive. But school improvement efforts can only persist if they are funded sustainably. How much does 
IPS invest in its restarts, and where does the funding come from? 

There are no continuing costs to IPS for most of the restarts themselves. The one exception is 
Phalen @ 103, where IPS pays an annual management fee of about 10 percent of per-pupil revenue. 
IPS will not pay management fees to other operators, so these costs will not grow. In addition, how- 
ever, philanthropic dollars contribute about $500,000 annually for the salaries and benefits of sev- 
eral district-level staff positions supporting Innovation Schools (including not only restarts, but new, 
charter, and conversion schools). The district plans to absorb these costs eventually, though it is not 


The other interventions included in this study also incur costs, though it is not always easy to pin 
down the amounts or sources, which may vary from year to year. For example, IPS has applied for and 
received state grant funding to support its Transformation schools. In 2018-19, this grant funding 
gave Transformation schools about $160,000 each. Similarly, Priority schools are eligible for some 
state and federal grants that target school improvement efforts. 

In addition, several schools in the Transformation Zone receive a “strategic supplement” each year 
from the district’s General Fund. The supplement is not specific to Transformation Zone schools, how- 
ever; any district school with a highly transient student population that is also undergoing an inter- 
vention is eligible. And as with Innovation schools, private funders have also supported Transforma- 
tion Zone schools financially, including paying for consulting support for some schools. Since those 
funds did not flow through the district, though, IPS has no way of tracking them. 

These examples lead to a few takeaways. First, most improvement efforts require some amount of 
additional funding, if only in the beginning. Second, private dollars often play a key role. However, it 
can be difficult to compare costs across initiatives because those costs are seldom tracked in a con- 
sistent way, and some dollars do not flow through the district. Ultimately, though, districts must be 
able to fund an initiative through regular per-pupil funding to sustain it over the long term. 


SUMMARY: RESTARTS AND STUDENT 
PERFORMANCE OUTCOMES 


%* Overall, the impact of restarts on the 
mean SGP of Stayers is unclear. Stayers 
demonstrated growth after the restart, 
but so did Leavers, and the difference 
between their growth was not significant 
after two years. 

%* Year-over-year growth has varied across 
restarts, but gains have been largest and 
most consistent at Phalen @ 103, where 
the restart has been in place longest. 


Discussion: Considering the 
Data in the Broader Context 


This study aimed to identify successes and lessons 
useful for future IPS school improvement efforts, as 
well as the efforts of others using or considering the 
restart model. Recognizing that the four restarts we 
analyzed for this study are all relatively new, we find 
the following: 


1. Enrollment trends offer positive signs. Increases 
in both reenrollment rates and overall enrollment 
suggest strong parent satisfaction with the restarted 
school. Moreover, the data show that the schools 

are mostly enrolling the same kinds of students 

they enrolled before the restart, suggesting that the 
intervention is largely reaching the students it was 


designed to help. 


2. Performance data from longest-running re- 
start suggest the possibility of large gains over 
time. The impact of IPS’s restarts on student per- 
formance have been mixed overall, with student 
growth improving in some schools and subjects, 

but not in others. However, the longest-running re- 
start—PLA @ 103—offers reasons to be optimistic. 
The schoolwide mean SGPs in both ELA and math 
have increased each year, for a change of more than 
15 percentiles in each subject compared with the 
three-year baseline. If PLA @ 103 can sustain these 
gains and other restarts can replicate them, they 
would provide strong evidence to support the restart 


model's efficacy in IPS. 


3. Anecdotal evidence highlights the difficulty of 
restarting struggling schools. Restarts are unlike 
most other school improvement efforts in that they 
bring an entirely new operator into an established 
school community. Hence, in addition to addressing 
the deep structural challenges a struggling school 
faces in instructional quality and school culture, the 
operators must also negotiate a major governance 
transition with teachers, parents, and students. 

The restarts at PLA @ 103, Global Prep, and Kin- 
dezi all encountered significant community concern, 
causing leaders to spend more time and energy than 
they anticipated in communicating with families and 


community partners (see “The Road to Restart,” 
pages 21-22). School leaders also had to recruit and 
hire far more teachers than they had hoped because 
only one of the original teachers from any of those 
schools reapplied for her job. 

Many charter operators acknowledge that dra- 
matically changing a school’s culture is much more 
difficult with hundreds of students in multiple grades 
than in a new school growing just one grade at a 
time. Both IPS and restart leaders have learned 
valuable lessons from these experiences—several 
of which improved the transition process for Ignite. 
Nonetheless, many still consider restarting a chroni- 
cally low-performing school to be much more diffi- 
cult than starting a school from the ground up, even 
if they serve students with the same level of need. 


4. Access to facilities gives districts a powerful 
enticement. If restarts are so difficult, then why 

did the operators in this study decide to take on the 
challenge? For one, they all have an unwavering 
commitment to serving the students of Indianapolis 
and meeting student needs wherever they exist. As 
these operators prepared to open new schools, those 
needs were greatest in the schools designated for 
restart. Moreover, all the restart operators had par- 
ticipated in paid, yearlong fellowships with The Mind 
Trust to help them lead Innovation schools, and The 
Mind Trust was now asking them to answer the dis- 
trict’s call for restarts. 

At the same time, the operators also noted that 
the restart model offered a precious commodity — 
access to a low-cost facility—and that without ac- 
cess to that facility, they may have not been able to 
open a school at all; with it, they were willing to pur- 
sue the difficult option of restart. 


5. Disconnected data systems hamper evaluation. 
For this project, the research team was fortunate 

to work with IPS and the Indiana Department of 
Education, which holds state test results for every 
public school student in the state, and to whom all 
operators must report many key metrics using a 
consistent format. Had the department denied the 
team’s data request, it would have had to establish 
data-sharing agreements with each of the dozens 
of charter operators in the city, and even then, the 


data would have likely arrived in a variety of forms. 
Similarly, IPS could not have conducted this analy- 
sis without also requesting data from the state or 
coordinating multiple charter operators. As districts 
cease to be the sole providers of public education in 
a city, clear gaps in data systems are emerging. 

As so often happens in studies like this one, the 
analysis also led to new questions requiring more 
information. For example, why did parents choose 
to enroll in the restarted schools if they did not yet 
have evidence of operator success? Similarly, why 
did other parents choose to enroll their children in a 
different school, and how did they decide where to 


enroll them? What are the longer term effects of the 
restarts on student outcomes, both for the students 
enrolled at the schools when the restart happened, 
as well as the students who enrolled (or chose to 
enroll elsewhere) over time? And what are the impli- 
cations of these findings on the way a district should 
approach their chronically low-performing schools? 
In the future, interviews with parents and students 
would provide important insights, in addition to ana- 
lyzing the same data over a longer period of time 
and additional student and school performance 


metrics. 


THE ROAD TO RESTART 


their voices into the restart process. 


core group of teachers. 


Changing Course at IPS 


The road to restart in IPS was a bumpy one at first. Restarts were a new tool, and the Innovation 
Network Schools model was a new governance structure. Many parents did not know what to make 
of the effort, and the district struggled with how best to communicate with parents and incorporate 


As aresult, the first restarts struggled to earn deep and authentic community buy-in. According 
to school leaders, many parents and teachers did not initially understand what it meant to be an In- 
novation school or why their school was targeted for restart, which in turn fueled misinformation. 
Many parents felt like the restart was something done to them, rather than something they chose, 
even when state takeover was the most likely alternative, school leaders said. And in the first three 
restarts, only one teacher reapplied for her job, a setback for operators who had hoped to retain a 


The new leader of a restarted school arguably plays the most influential role communicating with and 
reaching out to the school community during the transition period. The district also has a role, how- 
ever, and IPS recognized that it had misstepped along the way. After difficult rollouts in the first two 
rounds, it hired a family and community engagement manager to focus specifically on elevating com- 
munity voice in the restart process. Her initial work aimed to systematize parts of the restart process 
so that the rationale behind different decisions would be more transparent, to provide opportunities 
for community members to ask questions (and vent) early on so they could move quickly to problem- 
solving, and to work with community groups who could lead difficult but necessary conversations. 

As IPS began the process of restarting Elder Diggs 42, community investment was a primary focus 
for it and its restarting partner, Ignite Achievement Academy. Once the school community learned 
in January that the restart was likely, Ignite’s founders and IPS representatives began hosting town 
hall meetings at the school and at other community venues, like churches, and listening to parents, 
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THE ROAD TO RESTART (continued from previous page) 


community members, advocacy leaders, and others in the school community. Attendance grew with 
each meeting, allowing community members to ask questions and dispel misinformation. These ef- 
forts culminated with a community “Back to School Block Party” where dozens of families came out 
for fellowship, to meet the staff, and to enroll. 


Winning Neighborhood Support 


Meanwhile, leaders at the first three restarts worked to reintroduce themselves to the community 
and persuade families to consider their neighborhood schools. For example, PLA was announced as 
the new operator of Francis Scott Key @ 103 in March. In April, it hosted its first community event 
to offer families a chance to talk with school staff directly as rumors swirled in the neighborhood. 
Throughout the spring, PLA also held office hours from its space within 103 open to anyone. 

Over the summer, PLA improved the building, including mending fencing, replacing playground 
equipment, painting walls, and remodeling bathrooms. When the cosmetic work was done in July, 
PLA invited families back into the building for a “reveal,” showing how much was already different 
(and better) at the school. PLA then held another event in August to welcome students and their 
families back. 

Though the details differ at each school, the goal was the same: reset the school’s relationship 
with the community and show how good the restart was for students and families. 


Recommendations for District, 
Cities, and States 


It will take more time to know the full impact of IPS’s 
restarted Innovation schools. But even the first few 
years offer lessons for other districts, cities, and 
states adopting the restart model: 


1. Engage the school community early and hon- 
estly. Districts pursuing restarts should always put 
as much energy as possible into real and authentic 
community engagement from the very begin- 

ning. The experience of IPS and others repeatedly 
highlights the difference it makes when a school 
community supports a restart.”° Though commu- 
nity engagement will vary from place to place, the 
School Restart Authorization Process Guide outlines 
two main steps.”¢ First, conduct a broad community 
conversation about the vision for student success 
and commit to a comprehensive intervention for 
low-performing schools (including restarts) as a way 
to build support for and a sense of urgency around 
the need for dramatic change. Second, clearly define 
the role for community members in selecting a new 
operator. Members of the community do not need to 
choose the new operator, but the district gives them 
clear expectations and opportunities to share con- 
cerns and generate solutions. As the IPS example 
demonstrates, it is often best for the district to take 
a step back and let individuals and organizations 
that have credibility with families and the neighbor- 
hood lead this work. 


2. Use your assets. Examples abound of districts 
and charter operators at odds with each other.?” 
And many of the most contentious fights center on 
school siting and access to district facilities. IPS’s 
Innovation schools strategy represents a different 
approach: partnership. IPS needed high-quality op- 
erators to transform its lowest-performing schools, 
and it was willing to give them access to facilities at 


the same rate as its own schools to do so. Although 
this exchange may seem like an obvious solution, it 
is still an outlier nationally. Districts should take this 
opportunity to use their assets in partnering with 
nonprofit operators to meet their students’ needs. 


3. Build infrastructures that can support systems 
of schools across a city. Indianapolis is just one of a 
growing number of cities where nonprofit operators 
now enroll a substantial percentage of public school 
students. In 2016-17, for example, charter schools 
enrolled at least 20 percent of public-school stu- 
dents in nearly 60 cities.2® As more operators share 
responsibility for enrolling students in the same 
area, cities must update their infrastructures to 
serve students and families more seamlessly. More 
specifically, this report highlighted the need for data 
systems that allow districts, operators and other 
education champions to track students across differ- 
ent school types so they can identify what's working 
and what's not. 


Conclusion 


Chronically low-performing schools are an all-too- 
familiar challenge with too few solutions. Presented 
with the options of closing its lowest-performing 
schools or implementing a district turnaround, IPS 
instead proposed a new option—restart. Since then, 
charter operators have taken the helm of six schools 
the district once ran. 

At this point, it is not yet clear what impact IPS’s 
restarts will have on student performance outcomes. 
But the district’s first restart within the Innovation 
Schools Network offers reason for optimism, and 
even as policymakers wait to see how IPS's restarts 
perform over time, their experience offers keen in- 
sights and lessons for other districts interested in 
pursuing the same ‘third option.” 


Appendix A. Literature Review 


Most restart studies evaluate their impact on stu- 
dent performance, with mixed conclusions. Using 
matched samples for students in 11 charter restarts 
in New Orleans and one in Boston, a 2016 study 
concluded that charter restarts were more effec- 
tive than other less-aggressive improvement efforts, 
and that students who passively enroll in charters 
(by virtue of already being enrolled at the restarted 
school) benefit on a scale equal to students who 
apply to charters by lottery.2° Another 2016 study 
also matched students in restarted schools with 
similar students in other low-performing schools, 
and it too found that charter restarts in New Or- 
leans led to student gains.°° However, their research 
concluded that the opposite was true of restarts in 
Baton Rouge. The authors attributed the contrast to 
differences in the quality of the restarted school. 
Similarly, a three-year evaluation of Philadel- 
phia’s Renaissance Schools Initiative determined 
that most of the restarts operated by Mastery 
Charter Schools and ASPIRA Schools were on track 
to make dramatic improvements within five to six 
years, though performance at several other charter 
restarts was inconsistent.*! But an evaluation of 


Tennessee's Achievement School District using a 


difference -in-differences approach found that re- 
starts had either no effect on student achievement, 
or, in some cases, a negative effect.°? 

Restarts generally intend not just to improve stu- 
dent performance, but to do so for a very specific 
group of students—those enrolled in a failing school. 
Yet several studies examining student reenrollment 
in restarts show that reenrollment rates can vary 
widely. In a study of five charter-to-charter restarts, 
reenrollment rates ranged from 40 percent to 90 
percent.°? Similarly, reenrollment rates at a series of 
Chicago restarts that the Academy or Urban School 
Leadership (AUSL) operates ranged from $7 percent 
to 77 percent.*4 Meanwhile, Wolford et al. (2013) 
found that reenrollment in Philadelphia's charter re- 
starts was about 80 percent, and the percentage of 
English language learners decreased by a small but 
statistically significant amount, while the percent- 
age of students with an IEP increased by a small but 
statistically significant amount. Reenrollment rates 
were lower in New Orleans and Baton Rouge—48 
percent and 59 percent respectively—but the demo- 
graphics of students who reenrolled and those who 
did not were not substantially different (Bross 
et al., 2016). 


Appendix B. Reenrollment Rates by Student Subgroups 


Francis Scott Key 103 / Phalen Leadership Academy @ 103 


Percent of Eligible 


Percent of Reenrolled Students, 


Students, Baseline Beginning Year 1 Change 
Student Subgroup (N=299) (N=107) (Percentage Points) 
Black 80.7% 82.2% 1.5 
Hispanic 10.7% 11.2% 0.5 
Free & Reduced Lunch 87.9% 88.8% 0.0 
Limited English Proficiency 79% 9.3% 1.4 
Special Education 22.4% 11.2% -10.3 
Previously Retained 79% 8.4% 0.5 
Riverside 44 / Global Preparatory Academy 
Percent of Student Percent of Reenrolled Students, 
Body, Baseline Beginning Year 1 Change 
Student Subgroup (N=401) (N=257) (Percentage Points) 
Black 66.8% 65.4% -14 
Hispanic 24.2% 25.7% 1.5 
Free & Reduced Lunch 82.3% 85.6% 3.3 
Limited English Proficiency 12.5% 12.8% 0.3 
Special Education 14.0% 12.5% -1.5 
Previously Retained 2.6% 2.7% 0.1 
Joyce Kilmer 69 / Kindezi Academy 
Percent of Student Percent of Reenrolled Students, 
Body, Baseline Beginning Year 1 Change 
Student Subgroup (N=296) (N=177) (Percentage Points) 
Black 85.7% 88.1% 2.4 
Hispanic 9.8% 9.6% -0.2 
Free & Reduced Lunch 86.4% 84.7% -17 
Limited English Proficiency 3.5% 3.4% -0.1 
Special Education 12.9% 15.3% 2.4 
Previously Retained 7.3% 8.5% 1.2 
Elder Diggs 42 / Ignite Achievement Academy 
Percent of Student Percent of Reenrolled Students, 
Body, Baseline Beginning Year Change 
Student Subgroup (N=314) (N=202) (Percentage Points) 
Black 76.0% 82.2% 6.2% 
Hispanic 12.9% 9.9% -3.0% 
Free & Reduced Lunch 82.5% 86.6% 4.2% 
Limited English Proficiency 0.0% 0.0% 0.0% 
Special Education 22.2% 13.4% -8.9% 
Previously Retained 8.2% 7.9% -0.3% 
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FIGURE C1. PERCENTAGE OF BLACK STUDENTS, RESTARTS VS. COMPARISON SCHOOLS 
THREE YEARS PRE-RESTART THROUGH 2018-19 
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All Restarts 79.7% 74.0%* 5.7% 
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Priority Schools 60.1% 57.2% 
N=1925 N=2112 N=1933 


**Based on most recent year for which all schools have data (Y2), rather than an average of the rows above. 
**In 2018-19, one Transformation Zone school (Wendell Phillips) restarted as an Innovation school and another (George Washington Jr. High School) closed, accounting 
for much of the decline in enrollment across Transformation Zone schools that year. 
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FIGURE C2. PERCENTAGE OF BLACK STUDENTS ACROSS ALL PUBLIC SCHOOLS IN IPS BOUNDARIES, 2013-19 
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FIGURE C3. PERCENTAGE OF HISPANIC STUDENTS, RESTARTS VS. COMPARISON SCHOOLS, 
THREE YEARS PRE-RESTART THROUGH 2018-19 
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23.4% 25.1% 29.8% 
Priority Schools 
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*Based on most recent year for which all schools have data (Y2), rather than an average of the rows above. 
**In 2018-19, one Transformation Zone school (Wendell Phillips) restarted as an Innovation school and another (George Washington Jr. High School) closed, accounting 
for much of the decline in enrollment across Transformation Zone schools that year. 
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FIGURE C5. PERCENTAGE OF STUDENTS QUALIFYING FOR FREE OR REDUCED-PRICED LUNCH, 
RESTARTS VS. COMPARISON SCHOOLS, THREE YEARS PRE-RESTART THROUGH 2018-19 


Pre-Restart Post-Restart Change Over Time 
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*Based on most recent year for which all schools have data (Y2), rather than an average of the rows above. 
**In 201819, one Transformation Zone school (Wendell Phillips) restarted as an Innovation school and another (George Washington Jr. High School) closed, accounting for 
much of the decline in enrollment across Transformation Zone schools that year. 
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FIGURE C4. PERCENTAGE OF STUDENTS QUALIFYING FOR FREE OR REDUCED-PRICE LUNCH 
ACROSS ALL PUBLIC SCHOOLS IN IPS BOUNDARIES, 2013-19 
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FIGURE C7. PERCENTAGE OF STUDENTS WITH LIMITED ENGLISH PROFICIENCY, RESTARTS VS. COMPARISON SCHOOLS, 
THREE YEARS PRE-RESTART THROUGH 2017-18 
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*Based on most recent year for which all schools have data (Y2), rather than an average of the rows above. 
**In 2018-19, one Transformation Zone school (Wendell Phillips) restarted as an Innovation school and another (George Washington Jr. High School) closed, accounting 
for much of the decline in enrollment across Transformation Zone schools that year. 
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FIGURE C8. PERCENTAGE OF STUDENTS WITH LIMITED ENGLISH PROFICIENCY 
ACROSS ALL PUBLIC SCHOOLS IN IPS BOUNDARIES, 2013-19 
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FIGURE CQ. PERCENTAGE OF STUDENTS QUALIFYING FOR SPECIAL EDUCATION, RESTARTS V. COMPARISON SCHOOLS, 
THREE YEARS PRE-RESTART THROUGH 2018-19 


Pre-Restart Post-Restart Change Over Time 
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*Based on most recent year for which all schools have data (Y2), rather than an average of the rows above. 
**In 2018-19, one Transformation Zone school (Wendell Phillips) restarted as an Innovation school and another (George Washington Jr. High School) closed, accounting 
for much of the decline in enrollment across Transformation Zone schools that year. 
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FIGURE C10. PERCENTAGE OF STUDENTS QUALIFYING FOR SPECIAL EDUCATION 
ACROSS ALL PUBLIC SCHOOLS IN IPS BOUNDARIES, 2013-19 
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Appendix D. Historical Student Proficiency 


FIGURE D1. CHANGE IN MEAN STUDENT GROWTH PERCENTILES, STAYERS VS. LEAVERS, BASELINE YEAR TO YEAR 1, ELA 
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FIGURE D2. CHANGE IN MEAN STUDENT GROWTH PERCENTILES, STAYERS VS. LEAVERS, BASELINE YEAR TO YEAR 1, MATH 
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Note. “Stayers” defined as students eligible to reenroll at the end of the baseline year who did reenroll. To be included in the dataset, Stayers also had to have an SGP in 
the baseline year and Year 1. In addition, their SGP score in year 1 had to be linked to the restarted school. In contrast, “Leavers” were eligible to reenroll at the end of the 
baseline year, but did not reenroll. They also had to have SGP data for the baseline year and year 1, though their SGP in year 1 could not be linked to the restarted school. 
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FIGURE D3. CHANGE IN MEAN STUDENT GROWTH PERCENTILES, STAYERS VS. LEAVERS, BASELINE YEAR TO YEAR 2, ELA 


PLA @ 103 


Global Prep 


Tate (=¥ 41 


All Restarts 


Mean SGP Y2 


Change from baseline to Y1 


43.9 


43.6 


41.3 


44.8 


-16.9 


Diff-in- Diff-in- Diff-in- Diff-in- 
Stayers | Leavers Diff Stayers | Leavers Diff Stayers | Leavers Diff Stayers | Leavers Diff 
N=8 N=16 N=25 N=10 N=12 N=12 N=45 N=38 
Mean SGP Baseline 30.4 29.8 43.7 28.5 31.1 37.8 38.0 31.9 
Mean SGP Y1 46.4 59.8 56.4 58.1 35.8 50.0 49.1 56.2 


43.3 47.8 42.3 
4.8 


45.2 


p-value 


Change from Y1 to Y2 


0.245 
-1.8 


0.667 
7.5 


p-value 


Change from baseline to Y2 


0.907 


0.522 


p-value 


0.312 


0.180 


0.983 


0.462 


0.032* 


0.329 


PLA @ 103 


Global Prep 


Tate (=¥41 


FIGURE D4. CHANGE IN MEAN STUDENT GROWTH PERCENTILES, STAYERS VS. LEAVERS, BASELINE YEAR TO YEAR 2, MATH 
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Note. “Stayers” defined as students eligible to reenroll at the end of the baseline year who did reenroll. To be included in the dataset, Stayers also had to have an SGP in the 
baseline year, year 1, and year 2. In addition, their SGP scores in year 1 and year 2 had to be linked to the restarted school. In contrast, “Leavers” were eligible to reenroll 

at the end of the baseline year, but did not reenroll. They also had to have SGP data for the baseline year, year 1, and year 2, though their SGP scores in year 1 and year 2 
could not be linked to the restarted school. 
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